Multimodal Human-computer Interaction

Multimodal Human Computer Interaction: A Survey

In this paper, we review the major approaches to multimodal human–computer interaction, giving an overview of the field from a computer vision perspective. In particular, we focus on body, gesture, gaze, and affective interaction (facial expression recognition and emotion in audio). We discuss user and task modeling, and multimodal fusion, highlighting challenges, open issues, and emerging appl...

Multimodal Emotion Recognition for Human-Computer Interaction: A Survey

Today, the computer and its applications has invaded our daily life, Ubiquitous Computing. The interaction between the users and computing devices is becoming similar to human-human interactions. The integration of emotion recognition in Human-Computer Interaction aims at making the interaction easier and smarter, natural interaction. This paper, presents a brief survey on current state of the ...

Describing Multimodal Human-Computer Interaction

In this paper we argue that approaches to annotate multimodal human face-to-face interaction are not suitable for current device-based human-computer interaction. Instead, existing extensions proposed to established parameters describing the interaction with spoken dialog systems are presented and discussed. A standardization activity for annotating user interactions with multimodal systems is ...

Multimodal Human Computer Interaction

The MADE Framework: Multimodal Software for Affective Education

This paper proposes a framework for multimodal educational systems, considering the affective strategies. The ability to communicate emotionally and cognitively plays an important role in human-computer interaction (HCI) and education. The challenge is how theoretical models of HCI can inform affective multimodal education.

Chapter 25 : Multimodal interaction modeling

Virtual Environments (VE) represent advanced human computer interaction systems, communicating over several channels of information. The basic terms we need to describe human VE interaction are “communication channel” and “multimodality”. The communication channel is as a pathway between the user and the simulation that mediates the interaction. Multimodality is the quality of a system, which a...